Web Survey Bibliography
Relevance and research question: Cost pressures continue to necessitate switching single-mode designs, such as face-to-face (F2F), to inexpensive mixed-mode (MM) designs. This, however, involves the risk of finding different estimates in the MM and single-mode designs, if total survey error is affected by the redesign strategy. Differences in measurement and non-observation error are particularly relevant candidates for design differences in total error, also called measurement and selection effects (MEs and SEs). Knowledge about these effects is relevant to evaluate the usefulness of a MM redesign. We demonstrate the estimation of MEs and SEs for the case of the Dutch Crime Victimization Survey (CVS).
Methods and Data: We administered a split-ballot design, in which four independent samples (n=2,200 each) were drawn from the Dutch population register and assigned either to one of three sequential mixed-mode surveys (web, mail, and telephone, followed by F2F, respectively) or to a single-mode F2F condition, which served as benchmark. Additionally, the respondents to web, mail and telephone were approached a second time in F2F after four weeks. This step made available ‘within-subject’ estimates of MEs, which were exploited to disentangle MEs and SEs in the split-ballot design.
Results: Largest design differences in estimates were found for the mixed-mode mail-F2F design and, with smaller magnitude, also for web-F2F. The telephone-F2F survey showed mainly insignificant differences against the benchmark. In evaluating MEs and SEs, we found that MEs were the predominant cause of the differences between both, mail-F2F and web-F2F, and the benchmark, whereas SEs were generally very small. MEs and SEs were absent when comparing telephone-F2F against F2F.
Added Value: In the CVS case, the large MEs for the mail-F2F and web-F2F designs would require further redesign of questionnaires (e.g. by unimode strategies) to balance measurement error in mail and web towards F2F. Telephone-F2F yielded comparable estimates vis-à-vis the benchmark due to similar measurement error properties and could be implemented directly. The absence of SEs might suggest that the MM designs were successful in mitigating non-observation error differences between designs. More generally, our method could be used by other researchers to evaluate MM redesigns for other surveys.
Web survey bibliography (129)
- Social desirability bias in self-reported well-being measures: evidence from an online survey; 2017; Caputo, A.
- Comparability of web and telephone surveys for the measurement of subjective well-being; 2017; Sarracino, F.; Riillo, C. F. A.; Mikucka, M.
- Privacy Concerns in Responses to Sensitive Questions. A Survey Experiment on the Influence of Numeric...; 2016; Bader, F., Bauer, J., Kroher, M., Riordan, P.
- Thinking Inside the Box Visual Design of the Response Box Affects Creative Divergent Thinking in an...; 2016; Mohr, A. H.; Sell, A.; Lindsay, T.
- Detecting Insufficient Effort Responding with an Infrequency Scale: Evaluating Validity and Participant...; 2016; Huang, J. L.; Bowling, N. A.; Liu, Me.; Li, Yu.
- Detecting careless respondents in web-based questionnaires: Which method to use?; 2016; Niesen, A. S. M.; Meijer, R. R.; Tendeiro, J. N.
- Eye-tracking Social Desirability Bias; 2016; Kaminska, O.; Foulsham, T.
- Short and Sweet? Length and Informative Content of Open-Ended Responses Using SMS as a Research Mode; 2016; Walsh, E.; Brinker, J. K.
- Participant recruitment and data collection through Facebook: the role of personality factors; 2016; Rife, S. C.; Cate, K. L.; Kosinski, M.; Stillwell, D.
- Quantifying Under- and Overreporting in Surveys Through a Dual-Questioning-Technique Design. ; 2016; de Jong , M.; Fox, J.-P.; Steenkamp, J. - B. E. M.
- Will They Stay or Will They Go? Personality Predictors of Dropout in Online Study; 2016; Nestler, S.; Thielsch, M.; Vasilev, E.; Back, M.
- Development of a scale to measure skepticism toward electronic word-of-mouth; 2016; Zhang, Xia.; Ko, M.; Carpenter, D.
- Psychological research in the internet age: The quality of web-based data; 2016; Ramsey, S. R.; Thompson, K. L.; McKenzie, M.; Rosenbaum, A.
- Internet Abusive Use Questionnaire: Psychometric properties; 2016; Calvo-Frances, F.
- Equivalence of paper-and-pencil and computerized self-report surveys in older adults; 2016; Weigold, A.; Weigold, I. K.; Drakeford, M. K.; Dykema, S. A.; Smith, C. A.
- A multi-group analysis of online survey respondent data quality: Comparing a regular USA consumer panel...; 2016; Golden, L.; Albaum, G.; Roster, C. A.; Smith, S. M.
- Swapping bricks for clicks: Crowdsourcing longitudinal data on Amazon Turk; 2016; Daly, T. M.; Nataraajan, R.
- A reliability analysis of Mechanical Turk data; 2016; Rouse, S. V.
- Quota Controls in Survey Research.; 2016; Gittelman, S. H.; Thomas, R. K.; Lavrakas, P. J.; Lange, V.
- Exploring Factors in Contributing Student Progress in the Open University; 2016; Arifin, M. H.
- Incentive Types and Amounts in a Web-based Survey of College Students; 2015; Krebs, C.; Planty, M.; Stroop, J.; Berzofsky, M.; Lindquist, C.
- Comparison of telephone RDD and online panel survey modes on CPGI scores and co-morbidities; 2015; Lee, C.-K.; Back, K.-J.; Williams, Ro. J.; Ahn, S.-S.
- Enhancing Response Usability in a Web-based Survey, But Did Anyone Use It?; 2015; Yoder, R.
- Equivalency of Paper Versus Tablet Computer Survey Data; 2015; Ravert, R. D.; Gomez-Scott, J.; Donnellan, M. B.
- Development and Validation of a Scale for Social Exhibitionism on the Internet (SEXI); 2015; Vetter, M.; Eib, C.; Hill-Kloss, S.; Wollscheid, P.; Hagemann, D.
- Recruiting for addiction research via Facebook; 2015; Thornton, L. K.; Harris, K.; Baker, A.; Johnson, M.; Kay-Lambkin, F. J.
- Internet Research in Psychology; 2015; Gosling, S. D., Mason, W.
- The Prostate Cancer Journey Results of an Online Survey of Men and Their Partners; 2015; O'Shaughnessy, P. K., Laws, T. A., Esterman, A.
- Twelve-month prevalence and predictors of self-reported suicidal ideation and suicide attempt among...; 2015; Kang, E. H., Kim, G. M., Hyun, M. K., Choi, S. M., Kim, J. M., Woo, J. M.
- Emotion management in online groupwork reported by Chinese students; 2014; Xu, J., Du, J., Fan, X.
- Quality of physical therapy from a patient's perspective; factor analysis on web-based survey data...; 2014; Scholte, M., Calsbeek, H., Nijhuis-van der Sanden, M. W. G., Braspenning, J.
- Discriminating the Effects of Cannabis sativa and Cannabis indica: A Web Survey of Medical Cannabis...; 2014; Pearce, D. D., Mitsouras, K., Irizarry, K. J.
- Awareness and Treatment of Alcohol Dependence in Japan: Results from Internet-Based Surveys in Persons...; 2014; Taguchi, Y., Takei, Y., Sasai, R., Murteira, S.
- Awareness and correlates of the role of physical activity in breast cancer prevention among Japanese...; 2014; Miyawaki, R., Shibata, A., Ishii, K., Oka, K.
- Differences in intrapersonal and interactional empowerment between lurkers and posters in health-related...; 2014; Petrovcic, A., Petric, G.
- Evaluating mixed-mode redesign strategies against benchmark surveys: the case of the Crime Victimization...; 2014; Klausch, L. T., Hox, J., Schouten, B.
- Confirmation Bias in Web-Based Search: A Randomized Online Study on the Effects of Expert Information...; 2014; Schweiger, S., Oeberst, A., Cress, U.
- The Use of Paradata to Predict Future Cooperation in a Panel Study; 2014; Funke, F., Goeritz, A.
- The impact of New Zealand's 2008 prohibition of piperazine-based party pills on young people'...; 2013; Sheridan, J., Dong, C. Y., Butler, R., Barnes, J.
- PRM144 – An adaptable methodology for the design, implementation and conduct of a web-based survey...; 2013; Yeomans, K., Kawata, A. K., Bassel, M., Burk, C. T., Daniels, S. R., Wilcox, T. K.
- The internet user profile of Italian families of patients with rare diseases: a web survey; 2013; Tozzi, A. E., Mingarelli, R., Agricola, E., Gonfiantini, M., Pandolfi, E., Carloni, E., Gesualdo, F.,...
- Education in the Responsible Conduct of Research in Psychology: Methods and Scope; 2013; DiLorenzo, T. A., Becker-Fiegeles, J., Gibelman, M.
- Relative Mode Effects on Data Quality in Mixed-Mode Surveys by an Instrumental Variable; 2013; Vannieuwenhuyze, J. T. A., Revilla, M.
- Reaching and Hearing the Invisible: Organizational Research on Invisible Stigmatized Groups via Web...; 2013; Trau, R. N. C., Haertel, C. E. J., Haertel, G. F.
- On-line questionnaire completion time and personality test scores; 2013; Furnham, A., Hyde, G., Trickey, G.
- Why are you leaving me?? - Personality predictors of answering drop out in an online-study; 2013; Thielsch, M., Nestler, S., Back, M.
- The association between online gaming, social phobia, and depression: an internet survey; 2012; Chen, M.-H., Huang, P.-C., Bai, Y.-M., Wei, H.-T.
- P02.04. Internet survey confirms strong interest in Yoga among fibromyalgia patients; 2012; Carson, J., Bennett, R., Jones, K., Mist, S.
- GRE® program announces big benefits and big savings for GRE® test takers worldwide; 2011
- Going online with assessment; 2011; Burke, E., Mahoney-Phillips, J., Bowler, W., Downey, K.